United States Patent and Trademark Office 



UNITED STATES DEPARTMENT OF COMMERCE 
United States Patent and Trademark Office 
Address: COMMISSIONER FOR PATENTS * 
P.O. Box [450 

Alexandria, Virginia 22313-1450 
www.uspto.gov 



APPLICATION NO. 



FILING DATE 



FIRST NAMED INVENTOR 



ATTORNEY DOCKET NO. 



CONFIRMATION NO. 



09/585,980 



06/02/2000 



7590 



07/14/2004 



Dennis A Nicholls 
Blakely Sokoloff Taylor & Zafinan LLP 
12400 Wilshire Boulevard 7th Floor 
Los Angeles, CA 90025 



Lizhi Wang 



80398.P322 



5683 



EXAMINER 



HESSELTINE, RYAN J 



ART UNIT 



PAPER NUMBER 



2623 

DATE MAILED: 07/14/2004 



13 



Please find below and/or attached an Office communication concerning this application or proceeding. 



PTO-90C (Rev. 10/03) 



m 

Office Action Summary 


Application No. 

09/585,980 


Applicant(s) 

WANG, LI2HI 


exam 1 nor 

Ryan J Hesseltine 


Art Unit 

2623 





" The MAILING DATE of this communication appears on the cover sheet with the correspondence address - 
Period for Reply 



A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .1 36(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- tf the period for reply specified above Is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1 )I3 Responsive to connmunication(s) filed on 07 June 2004 . 
2a)n This action is FINAL. 2b)l3 This action is non-final. 

3) n Since this application is in condition for allowance except for fonnal matters, prosecution as to the merits is 

closed in accordance with the practice under £x parfe Quay/e, 1935 CD. 11,453 O.G.213. 

Disposition of Claims 

4) IE Claim(s) 1-45 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) 0 Claim(s) is/are allowed. 

6) M Claim(s) 1-45 is/are rejected. 

Claim(s) is/are objected to. 

8) n Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) n The specification is objected to by the Examiner. 

10) 0 The drawing(s) filed on is/are: aO accepted or b)n objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a). 
Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1 .121(d). 

11) 0 The oath or declaration is objected to by the Examiner. Note the attached Office Action orfonn PTO-152. 

Priority under 35 U.S.C. § 1 1 9 

12) n Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 119(a)-(d) or (f). 
a)n All b)n Some * c)\J None of: 

1 .□ Certified copies of the priority documents have been received. 

2. n Certified copies of the priority documents have been received in Application No. . 

3. n Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 



Attachment(s) 

1 ) CH Notice of References Cited (PTO-892) 

2) n Notice of Draftsperson's Patent Drawing Review (PTO-948) 

3) □ Information Disclosure Statement(s) (PTO-1449 or PTO/SB/08) 

Paper No(s)/IVIail Date . 



4) CH Interview Summary (PTO-413) 

Paper No(s)/Mail Date. . 

5) n Notice of Informal Patent Application (PTO-152) 

6) □ Other: . 



U.S. Patent and Trademark Office 

PTOL-326 (Rev. 1-04) 



Office Action Summary 



Part of Paper No./Mail Date 1 3 



Applicdtion/Control Number: 09/585,980 Page 2 

Art Unit: 2623 

DETAILED ACTION 
Response to Arguments 

1 . Applicant's arguments on pages 2-4, filed June 7, 2004, with respect to the rejection(s)of 
claim(s) 1, 12, 23, 34 and 45 under 35 U.S.C. § 103(a) have been fully considered and are 
persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, 
a new ground(s) of rejection is made in view of Lee et al. (USPN 6,542,625, previously cited). 

2. Applicants request for reconsideration of the finality of the rejection of the last Office 
action is persuasive and, therefore, the finality of that action is withdrawn. 

3. Applicant's arguments on page 3, filed June 7, 2004, with respect to claim 12, have been 
fully considered but they are not persuasive. On page 3, last paragraph, appHcant states, 
"Furthermore, in regards to claim 12, Applicant respectfully submits that the combination of 
Kinjo and Cosatto do not disclose or suggest, "a color gradient map of the color [graphic] 
image," "an intensity map of the color [graphic image]," or "a combined map configured to 
indicate true where said color gradient map is true and said [intensity] map is true," as claimed 
and described in the present patent appUcation." The examiner admits that Kinjo, Cosatto nor 
Lee (see below for new grounds of rejection) exphcitly discloses the recited "maps," but the 
examiner respectfully disagrees with applicant's statements. Kinjo discloses the use of a CCD 
image sensor 32 and an analog-digital (A/D) converter 36 (column 16, line 37-41), which are 
both well known in the art to produce a two-dimensional "image map." In this case, the 
examiner is reading step 158, wherein the respective pixels in the regions (image map) are 
differentiated along a plurality of mutually different directions wherein it is determined whether 
or not there is a region in which differential values change regularly, as "a color gradient map of 
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the color graphic image configured to indicate true where a color gradient has a low value" 
(column 18, line 65-column 19, line 17), and the examiner is reading stepl02, wherein the 
original image (map) is divided into a plurality of regions of black and white pixels by a 
predetermined threshold, as "an intensity map of the color graphic image configured to indicate 
true where an intensity value has a high value" (column 17, line 33-42). The examiner is further 
reading Lee's AND operation with respect to the detected color region and the image difference 
as "a combined map configured to indicate true where said color gradient map is true and said 
intensity map is true" (column 5, line 4-21). 

4. Applicant's arguments with respect to claims 3, 5, 14, 16, 25, 27, 36 and 38 have been 
considered but are moot in view of the new ground(s) of rejection. 

Claim Rejections - 35 JJSC § 103 

5. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 

obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

6. Claims 1, 2, 4, 8, 12, 13, 15, 19, 23, 24, 26, 30, 34, 35, 37, 41, and 45 are rejected under 
35 U.S.C. 103(a) as being unpatentable over Kinjo (USPN 5,629,752, previously cited) in view 
of Lee et al. (USPN 6,542,625, previously cited), hereafter Lee. 

7. Regarding claims 1, 23, and 45, Kinjo discloses an apparatus and method of determining 
at least one candidate patch for human faces in a color graphic image (column 19, line 18-25), 
comprising: determining a first area of the color graphic image wherein a color gradient 
(changing differential values) has a low value (exclude the regions in which differential values 
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change regularly; column 18, line 65 to column 19, line 17); determining a second area of the 
color graphic image (white region) wherein an intensity value has a high value (corresponding to 
a low pixel density value; column 17, line 33-42). 

8. Kinjo discloses that if a skin-colored region is present in an image and the hue and 
saturation of the skin-colored region (suitable hue saturation) are similar to that of the face of a 
human figure, there is the possibility of such a region being erroneously determined as 
corresponding to the face of a human figure (column 2, line 6-18), but does not disclose 
performing a logical AND on said first area and said and second areas to determine a third area 
of the color graphic image, or selecting portions of the third area with suitable hue saturation to 
form said at least one candidate patch. Lee discloses a method of detecting a specific object such 
as a human face in an image signal wherein the presence of both a color image difference 
indicating movement of a face and a color representing a characteristic of the object (such as a 
skin color) indicates that the face does appear (column 4, line 56-column 5, line 3). This 
operation is performed by executing an AND operation with respect to the detected color region 
and the image difference (column 5, line 4-21). It would have been obvious to one of ordinary 
skill in the art at the time the invention was made to perform a logical AND on a first area and a 
second area to determine a third area of a color graphic image as taught by Lee in order to 
improve the accuracy of face detection by using more than one characteristic to detect a face 
since the use of single characteristics such as skin color is not robust since it is subject to changes 
according to ethnic origin, illumination, background, etc. (column 1, line 48-58). Kinjo provides 
a similar argument in the discussion of only using skin color to detect facial regions since other 
skin-colored objects would be erroneously determined as face regions (column 2, line 6-18). 
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9. Regarding claim 12, Kinjo discloses a system configured to determine at least one 
location of a human face in a color graphic image (column 19, line 18-25), comprising: a color 
gradient map of the color graphic image configured to indicate true where a color gradient 
(changing differential values) has a low value (exclude the regions in which differential values 
change regularly; column 18, line 65 to column 19, line 17); an intensity map of the color 
graphic image configured to indicate true (white region) where an intensity value has a high 
value (corresponding to a low pixel density value; column 17, line 33-42). Lee (see above 
discussion of claims 1, 23, and 45) discloses a combined map configured to indicate true where 
said color gradient map (color image difference) is true and said intensity map (detected color 
region) is true (column 5, line 4-21), and at least one candidate patch is selected from said 
combined map, wherein said candidate patches have suitable hue (skin color) saturation (column 
4, line 60-column 5, line 3; see also Kinjo, column 2, line 6-18). 

10. Regarding claim 34, Kinjo discloses an apparatus comprising: a processor (facial-region 
extracting unit 40) coupled to a memory (ROM) through a bus (inherent); and a detection 
process (program) executed by the processor from the memory (column 16, line 37-48) to cause 
the processor to determine a first area of a color graphic image wherein a color gradient 
(changing differential values) has a low value (exclude the regions in which differential values 
change regularly; column 18, line 65 to column 19, line 17); determine a second area of the color 
graphic image (white region) wherein an intensity value has a high value (corresponding to a low 
pixel density value; column 17, line 33-42). Lee (see above discussion of claims 1, 23, and 45) 
discloses performing a logical AND on said first area and said second area to determine a third 
area of the color graphic image (column 5, line 4-21), and selecting portions of said third area 
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with suitable hue saturation (skin color) to form at least one candidate patch (column 4, line 60- 
column 5, line 3; see also Kinjo, column 2, line 6-18). 

1 1 . Regarding claims 2, 13, 24, and 35, Kinjo discloses that determining said first area uses a 
first threshold value comparison (column 39, line 36-44). 

12. Regarding claims 4, 15, 26, and 37, Kinjo discloses that determining said second area 
(density) uses a second threshold value comparison (column 17, line 33-42). 

13. Regarding claims 8, 19, 30, and 41, Kinjo discloses fitting an ellipse (elliptical region) to 
one of said at least one candidate patch (column 21, line 21-28). 

14. Claims 3, 5, 14, 16, 25, 27, 36 and 38 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Kinjo in view of Lee as applied to claims 2, 4, 13, 15, 24, 26, 35 and 37 above, 
and further in view of Lee et al. (USPN 5,583,659, previously cited), hereafter Lee '659. 

15. Regarding claims 3, 5, 14, 16, 25, 27, 36, and 38, Kinjo does not expHcitly disclose that 
said first or said second threshold is determined by normalization. On page 8, third paragraph, 
applicant defines normalization as the average value of the magnitude of the color gradient of all 
pixels, a fixed percentage of which is then selected as the threshold. Lee '659 discloses a multi- 
windowing technique for thresholding an image using local image properties wherein the 
averaged intensity value is used as a threshold level in determining the value of a center pixel 
(column 10, line 51 -column 11, line 21). It would have been obvious to one of ordinary skill in 
the art at the time the invention was made to determine the first and second thresholds by 
normalization as taught by Lee in order to provide a variable thresholding method relying on 
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localized image characteristics such as local averaged intensity to classify image pixels into 
either black object pixels or white background pixels (column 2, line 33-48). 

16. Claims 6, 7, 9-11, 17, 18, 20-22, 28, 29, 31-33, 39, 40, and 42-44 are rejected under 35 
U.S.C. 103(a) as being unpatentable over Kinjo in view of Lee as applied to claims 1,8, 12, 19, 
23, 30, 34, and 41 above, and further in view of Lobo et al (USPN 5,781,650, cited on 
applicant's IDS), hereafter Lobo. 

17. Regarding claims 6, 17, 28 and 39, Kinjo does not disclose that said third area is eroded 
or that said combined map includes an eroded boundary. Lobo discloses a system for automatic 
feature detection and age classification of human faces in digital images wherein a 
morphological operator first broadens the image's similar intensity regions, then narrows 
(erodes) the similar intensity regions in a copy of the image, and finally the narrowed image is 
subtracted from the broadened image (column 4, line 44-57). It would have been obvious to one 
of ordinary skill in the art at the time the invention was made to erode similar intensity regions as 
taught by Lobo in order to fit an oval (ellipse) to the face template using a potential image of an 
edge (column 4, line 44-57). 

18. Regarding claims 7, 18, 29, and 40, Lobo discloses that said eroding (narrowing) is 
morphological (column 4, line 49-63). 

19. Regarding claims 9, 20, 31, and 42, Kinjo does not expUcitly disclose determining if said 
ellipse is a bad fit to said at least one candidate patch or that said ellipse includes a degree of fit 
measure. Lobo discloses an oval fitting operation including iteratively updating the oval center 
position and the oval axes half-lengths until the total energy stabilizes around a minimum value 
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and a final fit has been reached (column 5, line 39-48). It would have been obvious to one of 
ordinary skill in the art at the time the invention was made to determine the degree of fit of an 
ellipse (oval) about a candidate patch as taught by Lobo in order to ensure that the ellipse/oval is 
properly positioned and scaled to include the optimum amount of information for personnel 
identification or the like (column 5, hne 28-38). 

20. Regarding claims 10, 21, 32, and 43, Lobo discloses further processing said at least one 
candidate patch when said eUipse is a bad fit (iteratively update until total energy stabilizes; 
column 5, line 39-48). 

21. Regarding claims 1 1, 22, 33, and 44, Kinjo discloses determining if said one of said at 
least one candidate patch is too smooth (column 18, line 54-63). 



Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Ryan J Hesseltine whose telephone number is 703-306-4069. 
The examiner can normally be reached on Monday - Friday, 8:30 AM - 5 PM. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Amelia Au can be reached on 703-308-6604. The fax phone number for the 
organization where this application or proceeding is assigned is 703-872-9306. 



Conclusion 
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Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 



Ryan J. Hesseltine 
June 29, 2004 



JINGGEWU 



